Agendas for multi-agent learning

Shoham et al. [1] identify several important agendas which can help direct research in multi-agent learning. We propose two additional agendas— called “modelling” and “design”—which cover the problems we need to consider before our agents can start learning. We then consider research goals for modelling, design, and learning, and identify the problem of finding learning algorithms that guarantee convergence to Pareto-dominant equilibria against a wide range of opponents. Finally, we conclude with an example: starting from an informally-specified multi-agent learning problem, we illustrate how one might formalize and solve it by stepping through the tasks of modelling, design, and learning. This report is an extended version of a paper which will appear in a special issue of Artificial Intelligence Journal [2]; in addition to the topics covered in that paper, this report contains several appendices providing extra details on various algorithms, definitions, and examples.